The present invention relates to providing for detection of scene illuminant and the use thereof to provide white balance correction in the digital photographic process.
To perform like the human visual system, imaging systems must automatically adapt to changing color casts in scene illumination. Simply put, white objects in a scene must be rendered as white, regardless of whether the scene illuminant was daylight, tungsten, fluorescent, or some other source. This process of automatic white adaptation is called xe2x80x9cwhite balancingxe2x80x9d and the corrective action determined by this adaptation mechanism is the white balance correction.
Automatic white balance algorithms employed in automatic printers, digital scanners, and digital cameras conventionally employ the digitized image information and related mathematical techniques to attempt to deduce from the image data the optimum level of white balance correction to be applied on a scene-by-scene basis to the image. It is known that errors in automatic white balance correction occur when the algorithm is unable to differentiate between an overall color cast caused by the scene illuminant and an overall color bias due to the composition of the scene. It is desirable, therefore, to be able to differentiate a color cast due to scene illumination from a color bias due to scene composition. It is also known that white balance errors occur due to color temperature variations within a class of scene illuminant. Late day direct sunlight imposes a yellowish color cast to a scene while skylight on a cloudy day will lend a bluish color cast to a scene. However, both lights are clearly daylight and will require substantially different white balance corrections. It is desirable, therefore, to also be able to account for scene illuminant color temperature variation when determining the white balance correction.
There are many methods described in the literature for determining the scene illuminant of a digital image. Some require special hardware at the time of image capture to make this determination. In commonly-assigned U.S. Pat. Nos. 4,827,119 and 5,037,198 a method of measuring scene illuminant temporal oscillations with the use of a dedicated sensor is described. Daylight will have no oscillation, while tungsten and fluorescent sources will fluctuate in output power due to the AC nature of their power supplies. The problem with any dedicated sensor approach is that it consists of two separate data collection and processing paths, one for illuminant detection and another for actual image capture. This leads to the potential of the dedicated sensor path losing synchronization and calibration with respect to the main image capture path. Additionally, the relatively limited amount of information captured by a dedicated sensor can severely limit the robustness of the scene illuminant determination. In commonly-assigned U.S. Pat. Nos. 5,644,358 and 5,659,357 the image data (video input) is combined with a luminance input to perform illuminant classification. (The nature of the luminance input is never described.) Rather than determining an overall illuminant for the scene, a low resolution version of the image is created and each image element (or xe2x80x9cpaxelxe2x80x9d) within the low resolution image is individually classified into one of a number of possible scene illuminants. Statistics are performed on these paxel classifications to derive a best compromise white balance correction. The problem with this approach is that no explicit attempt is made to uncouple the effects of scene illuminant color cast from the effects of scene composition. Instead, a complex series of tests and data weighting schemes are applied after the paxel classifications to try and reduce subsequent algorithm errors. Japanese Patent JP2001211458 teaches a method very similar to that described in commonly-assigned U.S. Pat. Nos. 5,644,358 and 5,659,357, and has the same problems. In commonly-assigned U.S. Pat. No. 6,133,983 scene-specific measurements of the scene light level, camera-to-subject distance, flash fire signal, and flash return signal are used to classify an image as being captured either under daylight or non-daylight illuminant. It is stated that for images captured with daylight-balanced films there is no need to further distinguish the non-daylight illuminants because the same white balance correction methodology works regardless. As a result, commonly-assigned U.S. Pat. No. 6,133,983 does not present a method for such subsequent illuminant discrimination. The problems with this approach are that it fails if applied to imaging systems requiring further differentiation of non-daylight sources for accurate white balancing, or if any of the image metadata (i.e., scene light level, camera-to-subject distance, flash fire signal, and flash return signal) are corrupt or missing.
There are many methods described in the literature for determining a color temperature responsive white balance correction of a digital image. In commonly-assigned U.S. Pat. Nos. 5,185,658 and 5,298,980 a method of measuring the scene illuminant""s relative amounts of red (R), green (G), and blue (B) power with dedicated sensors is described. The white balance correction values are derived from the ratios of RIG and B/G which are considered to be related to the color temperature of the scene illuminant. As with commonly-assigned U.S. Pat. Nos. 4,827,119 and 5,037,198, discussed above, the problem with any dedicated sensor approach is that it consists of two separate data collection and processing paths, one for illuminant detection and another for actual image capture, and these two paths can get xe2x80x9cout of stepxe2x80x9d with each other. In the above referenced JP200121458 the illuminant classification step is further refined to represent a variety of subcategories within each illuminant class. In this way cooler and warmer color cast versions of the illuminant classes of daylight, tungsten, and fluorescent are determined. However, as stated before, there is no explicit method given for uncoupling illuminant color cast from scene composition variability and, as a result, a variety of involved statistical operations are required in an attempt to minimize algorithmic errors.
It is therefore the object of the present invention to provide a simplified means for detecting and discriminating scene illuminants for use in computing white balance corrections and for the white balance corrections to be responsive to color temperature variations within each class of scene illuminants.
This object is achieved in a white balance correction method for a digital imaging device, comprising:
(a) determining a scene illuminant type from a scene brightness and one or more color coordinate values;
(b) determining a white balance parameter value responsive to the scene illuminant type, the scene brightness, and one or more color coordinate values;
(c) providing at least one white balance correction curve; and
(d) determining the white balance correction from the white balance parameter value and the white balance correction curve for the determined scene illuminant type.
It is an advantage of the present invention to provide a way of detecting and discriminating scene illuminants to effectively make white balance corrections.
Another advantage of the present invention is to provide an improved and simplified arrangement for providing white balance corrections.